Skip to content

fix: selectively copy items from hadoop-builder #1201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 11, 2025
Merged

fix: selectively copy items from hadoop-builder #1201

merged 2 commits into from
Jul 11, 2025
+30 −4

Conversation

@lfrancke
Copy link
Member

Description

fix: selectively copy items from hadoop-builder image to final image to not overwrite things from the base image

Definition of Done Checklist

Note

Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant.

Please make sure all these things are done and tick the boxes

  • Changes are OpenShift compatible
  • All added packages (via microdnf or otherwise) have a comment on why they are added
  • Things not downloaded from Red Hat repositories should be mirrored in the Stackable repository and downloaded from there
  • All packages should have (if available) signatures/hashes verified
  • Add an entry to the CHANGELOG.md file
  • Integration tests ran successfully
TIP: Running integration tests with a new product image

The image can be built and uploaded to the kind cluster with the following commands:

bake --product <product> --image-version <stackable-image-version>
kind load docker-image <image-tagged-with-the-major-version> --name=<name-of-your-test-cluster>

See the output of bake to retrieve the image tag for <image-tagged-with-the-major-version>.

@lfrancke lfrancke linked an issue Jul 11, 2025 that may be closed by this pull request
Since stackable-devel, some tools are missing #1200
Closed
@lfrancke
Copy link
Member Author

lfrancke commented Jul 11, 2025 •
edited
Loading

25.3 image for HDFS 3.4.1 looks like this:

stackable@618c1c2ae1d8 /stackable $ ll
total 10876
drwxrwx--- 1 stackable root     300 Mar 25 10:15 ./
drwxr-xr-x 1 root      root       0 Jul 11 10:48 ../
-rw-rw-r-- 1 stackable root      18 Feb 15  2024 .bash_logout
-rw-rw-r-- 1 stackable root     141 Feb 15  2024 .bash_profile
-rw-rw-r-- 1 stackable root     634 Mar 25 09:37 .bashrc
-rw-rw-r-- 1 stackable root    1791 Mar 25 09:37 .curlrc
-rw-rw-r-- 1 stackable root      47 Mar 25 09:37 .profile
drwxrwxr-x 1 stackable root      88 Nov 26  2022 async-profiler/
-rwxrwxr-x 1 stackable root 1168656 Mar 25 09:39 config-utils*
-rw-rw-r-- 1 stackable root   23863 Mar 25 09:39 config-utils.cdx.xml
-rwxrwxr-x 1 stackable root 9912456 Mar 25 09:42 containerdebug*
lrwxrwxrwx 1 stackable root      23 Mar 25 10:15 hadoop -> /stackable/hadoop-3.4.1/
drwxrwxr-x 1 stackable root     252 Mar 25 10:15 hadoop-3.4.1/
drwxrwxr-x 1 stackable root     152 Mar 25 09:35 jmx/
drwxrwxr-x 1 stackable root       6 Mar 25 09:42 vector/

0.0.0-dev image with this PR:

total 55756
drwxrwx--- 1 stackable root      428 Jul 11 11:17 ./
drwxr-xr-x 1 root      root        0 Jul 11 11:17 ../
-rw-rw-r-- 1 stackable root       18 Feb 15  2024 .bash_logout
-rw-rw-r-- 1 stackable root      141 Feb 15  2024 .bash_profile
-rw-rw-r-- 1 stackable root      634 Jul 11 10:22 .bashrc
-rw-rw-r-- 1 stackable root     1791 Jul 11 10:22 .curlrc
-rw-rw-r-- 1 stackable root       47 Jul 11 10:22 .profile
lrwxrwxrwx 1 stackable root       39 Jul 11 11:17 async-profiler -> /stackable/async-profiler-2.9-linux-x64/
drwxrwxr-x 1 stackable root        0 Nov 26  2022 async-profiler-2.9-linux-x64/
-rwxrwxr-x 1 stackable root  1168656 Jul 11 10:25 config-utils*
-rw-rw-r-- 1 stackable root    23863 Jul 11 10:25 config-utils.cdx.xml
-rwxrwxr-x 1 stackable root 12159184 Jul 11 10:26 containerdebug*
lrwxrwxrwx 1 stackable root       42 Jul 11 11:17 hadoop -> /stackable/hadoop-3.4.1-stackable0.0.0-dev/
drwxrwxr-x 1 stackable root        0 Jul 11 10:53 hadoop-3.4.1-stackable0.0.0-dev/
-rw-rw-r-- 1 stackable root 38810332 Jul 11 10:28 hadoop-3.4.1-stackable0.0.0-dev-src.tar.gz
-rw-rw-r-- 1 stackable root    89503 Jul 11 10:26 hdfs-utils-0.4.1-src.tar.gz
drwxrwxr-x 1 stackable root        0 Jun 11 19:16 jmx/
-rw-rw-r-- 1 stackable root     3472 Jul 11 11:17 package_manifest.txt
-rw-rw-r-- 1 stackable root  4799921 Jul 11 10:26 protobuf-3.7.1-src.tar.gz
drwxrwxr-x 1 stackable root        6 Jul 11 10:26 vector/

@lfrancke lfrancke marked this pull request as ready for review July 11, 2025 11:26
@lfrancke lfrancke enabled auto-merge July 11, 2025 11:26
@lfrancke lfrancke moved this to Development: Waiting for Review in Stackable Engineering Jul 11, 2025
NickLarsenNZ
NickLarsenNZ reviewed Jul 11, 2025
View reviewed changes
NickLarsenNZ
NickLarsenNZ reviewed Jul 11, 2025
View reviewed changes
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change requested

NickLarsenNZ
NickLarsenNZ approved these changes Jul 11, 2025
View reviewed changes
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lfrancke lfrancke added this pull request to the merge queue Jul 11, 2025
@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: Done in Stackable Engineering Jul 11, 2025
Merged via the queue into main with commit 24bd6e3 Jul 11, 2025
3 checks passed
@lfrancke lfrancke deleted the fix/hadoop-image branch July 11, 2025 11:38
@dervoeti dervoeti mentioned this pull request Jul 14, 2025
Merged
6 tasks
@lfrancke lfrancke moved this from Development: Done to Done in Stackable Engineering Jul 14, 2025
razvan pushed a commit that referenced this pull request Jul 22, 2025
@lfrancke @NickLarsenNZ @razvan
fix: selectively copy items from hadoop-builder (#1201)
a88d38d 
* fix: selectively copy items from hadoop-builder image to final image to not overwrite things from the base image

* Apply suggestion from @NickLarsenNZ

---------

Co-authored-by: Nick <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@NickLarsenNZ NickLarsenNZ NickLarsenNZ approved these changes

Assignees

No one assigned

Labels

None yet

Projects

Stackable Engineering
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Since stackable-devel, some tools are missing

3 participants

@lfrancke @NickLarsenNZ